Vehicle detection apparatus and vehicle detection method

ABSTRACT

A vehicle detection apparatus according to an embodiment comprises a controller detecting a vehicle from an image obtained by photographing the vehicle. The controller extracts a plurality of line-segment components indicating a boundary between a specific region of the vehicle and a vehicle body and included in the image. The controller measures a position of the vehicle based on coordinate information between the extracted line-segment components and photographing position information of the image.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a Continuation application of PCT Application No.PCT/JP2012/069171, filed Jul. 27, 2012 and based upon and claiming thebenefit of priority from prior Japanese Patent Application No.2011-170307, filed Aug. 3, 2011, the entire contents of all of which areincorporated herein by reference.

FIELD

Embodiments described herein relate generally to a vehicle detectionapparatus and a vehicle detection method.

BACKGROUND

For example, in tollgates of expressways, passage of a vehicle isgenerally detected by a pole sensor (a vehicle detection apparatus) of atransmission type and a reflection type using infrared laser.

However, there are various types of vehicles having different shapes,and the length from the distal end portion of the vehicle detected bythe pole sensor to a specific region (for example, a part around awindshield on which an on-board device is installed) differs for eachtype of vehicle. Thus, it is difficult to detect a specific region bythe pole sensor.

In addition, pole sensors require excavation in installment, and requireseparate equipment to adjust positions of the left and right sensors.Thus, pole sensors have a disadvantage of requiring high cost forconstruction and adjustment. Thus, pole sensors may be desirablyreplaced with vehicle detecting apparatuses of other means.

On the other hand, vehicle detection apparatuses using camerasrelatively easily achieve conditions for making the vehicle fall withina range of angle of view, by using existing pole sensors. The price ofcameras has fallen in recent years, and thus cameras are available at arelatively low price. It is thus preferable to achieve a vehicledetection apparatus using a camera.

A vehicle detection apparatus of a stereoscopic type using a pluralityof cameras will now be discussed. Generally, the feature points of avehicle differ according to the shape of the vehicle and how the vehiclelooks. Thus, in a stereoscopic system, the feature points that arecorrelated with one another between a plurality of images obtained by aplurality of cameras change each time, and the criteria for alignmentare indistinct.

As described above, vehicle detection apparatuses adopting astereoscopic system have a disadvantage that feature points that arecorrelated with one another between a plurality of images change eachtime, and the criteria for positioning are indistinct.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic diagram illustrating a configuration of a vehicledetection system, to which a vehicle detection apparatus according to afirst embodiment is applied.

FIG. 2 is a block diagram illustrating a configuration of the vehicledetection apparatus according to the first embodiment.

FIG. 3 is a schematic diagram illustrating a windshield region and linesegment components thereof in the first embodiment.

FIG. 4 is a flowchart for explaining operation according to the firstembodiment.

FIG. 5 (A) is a diagram illustrating a stereoscopic camera model in thefirst embodiment, and FIG. 5 (B) is a diagram illustrating a relationbetween a camera coordinate system and a global coordinate system.

FIG. 6 (A) is a schematic diagram illustrating rough approximation of awindshield in a second embodiment, and FIG. 6 (B) is a schematic diagramillustrating fine approximation thereof.

FIG. 7 is a flowchart for explaining operation according to the secondembodiment.

FIG. 8 is a diagram for explaining operation of measuring a vehiclewidth and a vehicle height, performed by a vehicle detection apparatusaccording to a third embodiment.

DETAILED DESCRIPTION

One embodiment is to provide a vehicle detection apparatus that canclarify the criteria for alignment by determining feature points thatare correlated with one another between a plurality of images, andimprove vehicle detection accuracy. According to one embodiment, avehicle detection apparatus according to an embodiment comprises adetecting module detecting a vehicle from an image obtained byphotographing the vehicle, an extracting module, and a measuring module.The extracting module extracts a plurality of line-segment componentsindicating a boundary between a specific region of the vehicle and avehicle body and included in the image. The measuring module measures aposition of the vehicle based on coordinate information between theextracted line-segment components and photographing position informationof the image.

Embodiments will now be explained hereinafter, with reference todrawings.

A matter common to the embodiments is a configuration for extracting awindshield region from left and right camera images, and in particularthe windshield region (a polygonal approximate region) of the vehicle.Another matter common to the embodiments is a configuration that enablesa stereoscopic view of the vehicle, and achieving highly-accuratevehicle detection, by correlating the left and right notedcharacteristics with each other, using line-segment components formingthe windshield region as characteristic amounts.

First Embodiment

FIG. 1 is a schematic diagram illustrating a configuration of a vehicledetection system, to which a vehicle detection apparatus 100 accordingto the present embodiment is applied. As illustrated in FIG. 1, thevehicle detection system comprises a pole sensor 10, an electroniccamera 20, an ETC (Electronic Toll Collection) system 30, and thevehicle detection apparatus 100.

The pole sensor 10 detects a vehicle 40 entering an ETC lane by using anoptical sensor or a tread sensor, and notifies the vehicle detectionapparatus 100 of a result of the detection.

The electronic camera (hereinafter simply referred to as “camera”) 20 isformed of left and right digital cameras to photograph the vehicle 40 inleft and right directions with respect to a running direction of thevehicle 40. Specifically, the digital cameras in the camera 20 areinstalled in a line in the lateral direction and in a position above andin front of the running direction of the vehicle 40. Each of the digitalcameras takes a moving image of the vehicle 40 running on the ETC laneand passing through the pole sensor 10, at a preset frame rate. Thus,the camera 20 takes and outputs a plurality of images for the vehicle 40running on the ETC lane.

In the following explanation, a windshield is referred to as an exampleof a specific region of the vehicle 40. Thus, the camera 20 is installedin a position where it can shoot a panoramic view at least including thewindshield of the vehicle 40. In addition, the camera 20 is installed toinclude the windshield and side surfaces of the vehicle 40 in avisual-field area 200 as a photographing position. Each of the digitalcameras of the camera 20 may be arranged side by side along a directionalmost parallel with the longitudinal direction of upper and lower sidesof the windshield of the vehicle 40. For example, there are cases wherethe vehicle 40 enters or leaves the ETC lane inclined with respect tothe optical axis of the camera 20. In such a case, line-segmentcomponents indicating the boundary between the windshield region and thevehicle body of the vehicle 40 are uniformly inclined, and thus the leftand right digital cameras may be installed inclined in a directionorthogonal to a representative line-segment component thereof.

The camera 20 may be configured to optically acquire left and rightimages of the vehicle 40 with a lens, instead of the left and rightdigital cameras. For example, the camera 20 may have a structure capableof simultaneously branching images from two directions with speciallenses of a camera. Specifically, the camera is configured to use awedge prism polarization system in which two lenses are arranged at, forexample, an inlet of light, and a prism that bends the optical path isdisposed inside.

The camera 20 transmits image data including a time code that indicatesthe photographing time to the vehicle detection apparatus 100. Thecamera 20, the vehicle detection apparatus 100 and the other device (30)have synchronized time information. The image data output from thecamera 20 may not include any time code indicating the photographingtime, in the case of adopting a structure in which the camera 20 and thevehicle detection apparatus 100 and the other device operate insynchronization, and the vehicle detection apparatus 100 and the otherdevice can recognize the photographing time of the camera 20.

The ETC system 30 is a system configured to automatically collect a tollfor the vehicle 40 running on a toll road such as an expressway. The ETCsystem 30 obtains information for identifying the passing vehicle 40, bywireless communication with an ETC in-vehicle unit mounted on thevehicle 40. Generally, an ETC in-vehicle unit has an antenna forperforming wireless communications, and the antenna is installed in aposition which can be visually recognized through the windshield of thevehicle 40. Thus, since the position of the windshield is accuratelyspecified, the ETC system 30 can perform highly accurate wirelesscommunication with the ETC in-vehicle unit.

Next, as illustrated in FIG. 2, the vehicle detection apparatus 100includes a display module 110, a user interface 120, a storage module130, a network interface 140, and a controller 150.

The display module 110 is a display device using an LCD (Liquid CrystalDisplay) or the like, and displays various information items such as anoperation state of the vehicle detection apparatus 100. The userinterface 120 is an interface that receives user instructions via inputdevices such as a keyboard, a mouse, and a touch panel. The storagemodule 130 stores control programs and control data for the controller150, and is formed of one or a plurality of storage devices, such asHDDs (hard disk drive), RAMS (random access memory), ROMs (read onlymemory), and flash memories. The network interface 140 is connected to anetwork such as a LAN, and communicates with each of the pole sensor 10,the camera 20, and the ETC system 30 through the network.

The controller 150 includes a microprocessor, and operates based on thecontrol programs and control data stored in the storage module 130. Thecontroller 150 serves as a supervising controller for the vehicledetection apparatus 100. The controller 150 detects a specific region ofthe vehicle 40 that is set with the control data in advance, based onthe image obtained by the camera 20, and executes processing ofspecifying the position of the vehicle 40 in real space. The controller150 may execute processing of predicting the passage time (the time atwhich the vehicle 40 passes through the communication area of the ETCsystem 30) in real space, together with specifying the position of thevehicle 40.

The controller 150 has the following functions (f1) to (f4), to executethe processing of specifying the position of the vehicle 40.

The function (f1) is a vehicle detecting function of detecting entry ofthe vehicle 40 from the left and right images obtained from the camera20 formed of left and right digital cameras. Background subtraction andpattern matching are used for the vehicle detecting function. However,the vehicle detecting function of the camera 20 can be omitted, sincethe object detected by the pole sensor 10 is the vehicle 40 in manycases.

The function (f2) is a line-segment component extracting function ofextracting a plurality of line-segment components indicating theboundary between the windshield region and the vehicle body of thevehicle 40 and included in the image, for each of the left and rightimages obtained by photographing the vehicle 40. Specifically, forexample, the line-segment component extracting function is a function ofextracting a plurality of line-segment components e1 to e6 indicatingthe boundary between the windshield region and the vehicle body asillustrated in FIG. 3 (B), for a photograph image illustrated in FIG. 3(A).

The function (f3) is a polygon approximation function of performingapproximation with a polygon forming a closed loop using some of theline-segment components extracted from the image, for each of the leftand right images. As a supplementary explanation, the closed loopcorresponding to the windshield region is formed of a combination of theline-segment components e1 to e6 extracted by the line-segment componentextracting function. The line-segment component extracting function andthe polygon approximation function can be achieved by, for example, themethod disclosed in a Japanese Patent Application Publication (Jpn. Pat.Appln. No. 2010-208539).

The function (f4) is a function of aligning at least one line-segmentcomponent, in the line-segment components e1 to e6 forming the closedloop for each of the left and right images, with each other between theleft and right images. In addition, the function (f4) is a measuringfunction of measuring the position of the closed loop as the position ofthe vehicle, based on coordinate information between the alignedline-segment components and photographing position information (theposition where the camera 20 is placed) indicating the photographingposition where the left and right images are photographed.

The measuring function of the function (f4) may include processing ofaligning all the line-segment components forming the closed loop. Forexample, the processing involves matching all the windshield regions ofthe left and right images to correlate the left and right representativepoints with each other, in the case where the windshield of the vehicle40 has many curved faces and it is difficult to determine arepresentative line-segment component (at least one line-segmentcomponent).

Next, operation of a vehicle detection system, to which the vehicledetection apparatus 100 of the present embodiment is applied, will beexplained hereinafter with reference to the flowchart of FIG. 4.

When the vehicle detection apparatus 100 is powered and started, thevehicle detection apparatus 100 repeatedly executes the steps (ST2 toST5) illustrated in FIG. 4 until the power is turned off. These stepsare executed by operation of the controller 150.

Prior to startup of the vehicle detection apparatus 100, the pole sensor10 and the camera 20 are started. Thereby, the pole sensor 10 startsmonitoring entry of the vehicle 40 into the ETC lane, and notifies thevehicle detection apparatus 100 of a detection result, until the poweris turned off. The camera 20 starts photographing at a predeterminedframe rate, and transmits the obtained image data to the vehicledetection apparatus 100 until the power is turned off (ST1). Thereby,the controller 150 receives image data obtained by the camera 20.

The controller 150 executes the vehicle detecting function ofdetermining whether the object detected by the pole sensor 10 is avehicle or not, in response to notification from the pole sensor 10through the network interface 140. Specifically, the controller 150detects entry of the vehicle 40, based on results of backgroundsubtraction processing and pattern matching processing performed for theleft and right images obtained from the camera 20 (ST2).

The left and right digital cameras of the camera 20 are placed such thatthe windshield region can be photographed with at least a plurality offrames for the vehicle 40 running in the angle of view of the camera.The controller 150 presets on the screen an observed region, for whichit is determined whether a vehicle has entered or not, and detectschanges in image in the region by background subtraction or the like.The controller 150 determines whether the detected object is a vehicleor not, by setting a part including the observed region as a searchregion, and matching the search region with a vehicle front pattern(lights, front grille, and number plate) prepared in advance. As anothermethod, a number plate being a characteristic of the vehicle may bedetected.

When entry of the vehicle 40 is detected, the controller 150 goes to thestep of extracting line segments of the windshield region as follows(YES of ST2, and ST3). When no entry of vehicle 40 is detected, thecontroller 150 continues monitoring of vehicle entry (NO of ST2).

The controller 150 extracts an image data item of a frame photographedat a predetermined time among a plurality of image data itemstransmitted from the camera 20, through the network interface 140. Theextracted image data item is referred to as image data to be processed.The predetermined time is determined in consideration of the positionalrelation (distance) between the position where the pole sensor 10 isinstalled and the angle of view (photographing range) of the camera 20,and the assumed passing speed of the vehicle, such that image dataincluding the specific region of the vehicle 40 can be extracted.

The controller 150 performs preprocessing for the image data to beprocessed. Specifically, the preprocessing includes noise reductionperformed for the purpose of improving the S/N ratio. The image issharpened by the noise reduction. The preprocessing also includesfiltering to improve image contrast. The preprocessing also includesimage distortion correction for the purpose of correcting the image.

The controller 150 extracts a plurality of line-segment components e1 toe6 indicating the boundary between the windshield region and the vehiclebody of the vehicle 40, from the image data to be processed that hasbeen subjected to preprocessing, using a method such as Hough transform(ST3). Specifically, the controller 150 extracts a plurality ofline-segment components e1 to e6 indicating the boundary included in theimage, for each of the left and right images, for which entry of thevehicle 40 has been detected.

As a specific extraction algorithm of Step ST3, for example,eight-direction line-segment components based on horizontal and verticaldirections in the image are extracted, when the vehicle 40 isphotographed from above. Thereby, a number of line-segment componentsincluding a windshield boundary part are extracted. Since a part of awindshield around wipers is curved in many cases, it is difficult toextract the boundary with one line-segment component. Thus, generally,it is possible to perform processing of approximating the shape of thewindshield by extracting the boundary as a polygon or lines obtained bycombining a plurality of line-segment components. For example, when acircle is approximated with line-segment components, it can beapproximated with an inscribed equilateral octagon. An error in thiscase corresponds to a difference in area between the circle and theinscribed equilateral octagon, and is permissible as an error inpractical design.

The controller 150 performs Hough transform for the image data to beprocessed and image data items of previous and following frames, andextracts time-successive line-segment components from the images. Thecontroller 150 may execute geometric change prediction with movement ofthe vehicle 40 from the line-segment components, and obtain line-segmentcomponents at the predetermined time (the photographing time of theimage data to be processed). Using image data items of a plurality offrames like this can improve the extraction accuracy.

Although the processing of extracting the line-segment components usinga method such as Hough transform has been explained, when colorinformation is included in the image data to be processed, the imagebased on the image data to be processed may be divided into regions ofsimilar colors based on the color information, and boundaries betweenthe regions may be extracted as line-segment components. Also by such amethod, a boundary between the windshield and the other regions of thevehicle can be extracted as line-segment components.

The controller 150 executes approximation with a polygon that forms theclosed loop using some of the line-segment components extracted in StepST3 for each of the left and right images, and generates candidates forthe windshield region. Elements extracted from the image by polygonapproximation are the windshield region, a shadow region cast on thewindshield, a reflection region including reflected light from the sun,pillars being a part of the vehicle, and windows for the driver's seatand the front seat next to the driver. Actually, a closed loop with acomplicated shape is formed with a plurality of line-segment components.

Although the simplest shape of the windshield region can be approximatedto a rectangle, the windshield region can be approximated to a shapeincluding curves according to the shape of the windshield. Even if thewindshield has a simple shape, when it is photographed from the side ofthe vehicle, a depth occurs between the left and right sides of thewindshield and asymmetry is generated.

At this point in time, it has not been recognized which line-segmentcomponent is a part of the windshield region. An optimum solution existsin a combination of closed loops including the line-segment componentsindicating the boundary between the windshield and the vehicle body bythe polygon approximation. Thus, the controller 150 performs evaluationfor a plurality of closed loops generated by polygon approximation,using an evaluation function, and narrows down the candidates to onethat accurately approximates the windshield region.

Actually, there may be parts with high curvature or parts withinsufficient contrast of boundaries, and there may be candidates thatpartly lack line-segment components. Thus, the evaluation may beperformed after performing approximation to supplement the loss partwith a line-segment component. For example, according to thephotographing angle, one of the pillars may be concealed by thewindshield. In such a case, the windshield end part on the side of theconcealed pillar is supplemented with a line-segment component, tocomplete the closed loop.

In addition, various pillar patterns are stored in advance in thestorage module 130, and a plurality of windshield region candidates arecorrelated with each pattern and stored in the storage module 130. Then,a closed loop similar to the pillar may be detected from polygonapproximation, the pillar may be detected by pattern matching betweenthe detected closed loop and the information stored in the storagemodule 130, and windshield region candidates correlated with thedetected pillar may be obtained.

The controller 150 performs a plurality of different evaluations foreach of the windshield region candidates obtained as described above,and determines a total of scores of the evaluations for each windshieldregion candidate. The evaluations include a first method of providingthe candidates with scores in consideration of the position and size ofthe windshield in the image. The evaluations include a second method ofproviding the candidates with scores based on brightness distributionsaround the line-segment components forming the windshield region. Theevaluations also include a third method of providing the candidates withscores based on the degree of matching with a windshield template storedin advance in the storage module 130. When a polygon appears in thewindshield region due to reflection or shadows, the polygon is providedwith low scores by the first method and the third method.

The controller 150 selects an optimum windshield region based on thetotals of the scores as described above. The controller 150 checks thepositional relation between the selected windshield region and a frontmask part (lights, grille, number plate) included in the image data tobe processed, and examines whether there are any contradictions (forexample, whether there is a large shift in a lateral direction betweenthe windshield region and the front mask part). When there is acontradiction, the same examination is performed for the windshieldregion having the second highest total score. The position of thevehicle front mask part is determined by pattern matching of elementsforming the front mask part.

Next, when the controller 150 extracts a plurality of line-segmentcomponents indicating the boundary between the windshield region and thevehicle body for each of the left and right images, the controller 150performs processing of aligning at least one line-segment componentbetween the left and right images (ST4). Specifically, the controller150 aligns at least one line-segment component among line-segmentcomponents forming a closed loop for each of images of the image data tobe processed between the left and right images.

Thereafter, the controller 150 measures the position of the closed loopas the position of the vehicle 40, based on coordinate informationbetween the aligned line-segment components and photographinginformation indicating the photographing position where the left andright images are photographed (ST5).

Specifically, as illustrated in FIG. 5 (A) and FIG. 5 (B), thecontroller 150 converts camera coordinates (coordinates on a virtualimage plane) obtained by obtaining a difference between coordinateinformation items into a global coordinate system (position in realspace on the ETC lane), and thereby calculates the position of theline-segment components of the windshield region. The calculatedposition corresponds to the position of the vehicle to be measured inStep ST5.

Reference symbols C₁ and C₂ illustrated in FIG. 5 (A) indicatephotographing positions (intersection points between the X axis of thephotographing plane coordinate system and the optical axes of the leftand right cameras) of the left and right cameras in the cameracoordinate system using an intermediate point between the left and rightcameras of the camera 20 as an origin O. The left and right cameras aretwo cameras having the same specifications, and placed in parallel witheach other and at equivalent positions. Thus, the optical axes of thetwo cameras are made parallel so that their imaging planes agree witheach other, and so that the horizontal axes of the imaging planes agreewith each other. A reference symbol b indicates an interval between theleft and right cameras. A reference symbol f indicates a focal distance.Reference symbols [x_(l), y_(l)] and [x_(r), y_(r)] are coordinates ofpoints (such as end points) corresponding to each other in the alignedline segment in imaging plane coordinate systems on the left and rightimages (“Image Plane” in the drawing), using intersection points withthe optical axes (broken-line arrows in the drawing) of the right andleft cameras as origins.

Objects [X, Y, Z] are coordinates of the corresponding points in thecamera coordinate system, and have the following relation.Z=b·f/(x _(l) −x _(r))X=Z·x _(r) /fY=b·y _(r) /f

Reference symbols R and t illustrated in FIG. 5 (B) are externalparameters to associate the camera coordinates with the globalcoordinates. The reference symbol R indicates a rotation matrix.Reference numeral t indicates a parallel movement vector.

The controller 150 notifies the ETC system 30 of the vehicle position(coordinates) measured in Step ST5 through the network interface 140.The ETC system 30 transmits and receives wireless signals to and from anantenna of the ETC in-vehicle unit mounted on the windshield, based onthe position (coordinates) of the vehicle 40 and the assumed passingspeed of the vehicle 40.

As described above, according to the present embodiment, the controller150 aligns at least one of line-segment components of the left and rightimages among line-segment components forming a closed loop usingline-segment components indicating the boundary between the windshieldregion and the vehicle body of the vehicle 40. Thereby, the alignmentstandard can be clarified by determining feature points that are to becorrelated with each other between images from the camera 20, and thusthe accuracy of detecting the vehicle 40 can be improved.

Specifically, line-segment components of the vehicle windshield regionboundary part are extracted from the left and right camera images, therepresentative line-segment components of the left and right images arecorrelated with each other, and thereby three-dimensional measurement ofthe vehicle 40 can be achieved with high accuracy. In addition, when itis applied to the ETC system 30, the ETC in-vehicle unit is placed inthe vicinity of the windshield, and thus the positional relation withthe in-vehicle unit to communicate with and the distance from thein-vehicle unit can be accurately extracted. Further, informationcorresponding to the width and the height of the vehicle can bedeductively determined, based on positional information of thewindshield.

In addition, according to the present embodiment, the controller 150extracts a plurality of line-segment components forming a vehicle imagefrom image data obtained by photographing the vehicle, and executesapproximation with a polygon for generating a closed loop by using theline-segment components. Thereby, the controller 150 generates aplurality of candidates for a region of a specific part (for example,windshield) of the vehicle 40, performs a plurality of differentevaluations for the candidates, and specifies a most probable vehiclespecific part region. Thus, since the above specific part can bedetected by image processing as long as the specific part of the targetvehicle 40 is imaged, the camera 20 can be installed with a high degreeof freedom, and a high detection accuracy can be obtained.

Second Embodiment

FIG. 6 and FIG. 7 are diagrams for explaining a second embodiment. Avehicle detection apparatus 100 and a vehicle detection system accordingto the present embodiment are the same as those of the first embodimentexplained with reference to FIG. 1 and FIG. 2, and thus explanationthereof will be omitted.

In the present embodiment, a controller 150 has a configuration thatincludes a re-executing function (f2-1) in a line-segment extractingfunction (f2) to extract line-segment components of a windshield region.

The re-executing function (f2-1) is a function of classifying extractedline-segment components according to directions, and re-executing theprocessing of extracting a plurality of line-segment components based onthe number and length of the line-segment components for each direction.Specifically, as illustrated in FIG. 6 (A) and FIG. 6 (B), there-executing function (f2-1) includes changing parameters used inextraction of the line-segment components to increase the number of theline-segment components and reduce the length of the line-segmentcomponents, and re-executing the processing of extracting theline-segment components based on the changed parameters.

By adding the re-executing function (f2-1) as described above,parameters such as the size of the edge filter are changed in theprocessing of extracting line-segment components being constituentelements of the windshield region, and thereby the extraction rate forthe line-segment components can be improved. The extraction rate for theline-segment components can be determined based on an extraction result(the number and length of line-segment components for each direction).The extraction rate for the line-segment components can be calculated asa proportion of the actual extraction result in the expected extractionresult, with respect to the number and length of line-segment componentsfor each direction.

In addition, as illustrated in FIG. 6 (A), when the size of thewindshield region is greater than a reference size, the re-executingfunction (f2-1) reduces the number of line-segment components andincreases the length of the line-segment components. On the other hand,as illustrated in FIG. 6 (B), when the size of the windshield region isequal to or less than the reference size, the re-executing function(f2-1) may be configured to include a function of changing theparameters used in extraction of line-segment components to increase thenumber of the line-segment components and reduce the length of theline-segment components.

Specifically, when there is a large difference in the size of thewindshield, such as windshields of compact cars and windshields oflarge-sized buses, processing to change the resolution for the object tobe photographed may be performed according to the resolution for thephotographed windshield, such that rough approximation is executed forthe windshield region with a camera with high resolution and fineapproximation is executed for the windshield region with a camera withlow resolution. Distinction between large-sized cars and small cars canbe made by extracting headlight regions of the vehicle 40 by labelingprocessing or the like from the image data to be processed afterpreprocessing, and estimating the size of the vehicle based on the leftand right headlight positions and an interval between the left and rightheadlights indicated by the extracted regions.

Next, operation of the vehicle detection apparatus 100 and the vehicledetection system according to the present embodiment will be explainedhereinafter, with reference to the flowchart of FIG. 7.

First, in the same manner as the first embodiment, a camera 20 transmitsphotographed image data to the vehicle detection apparatus 100 (ST11).The controller 150 of the vehicle detection apparatus 100 receives imagedata photographed by the camera 20. The controller 150 executes avehicle detecting function and detects entry of the vehicle 40 (YES ofST12). Then, when the controller 150 detects entry of the vehicle 40,the controller 150 extracts a plurality of line-segment componentsindicating a boundary between the windshield region and the vehicle bodyof the vehicle 40 (ST13). When the controller 150 detects no entry ofthe vehicle 40, the controller 150 continues monitoring of vehicle entry(NO of ST12).

Next, the controller 150 classifies the extracted line-segmentcomponents according to directions, and determines whether an extractionresult is good or not, based on the number and length of line-segmentcomponents for each direction (ST15). The determination is performedbased on whether an extraction rate of the line-segment components ishigher than a predetermined extraction rate or not.

When the extraction result is not good, the controller 150 changesparameters used in extraction of line-segment components, to increasethe number of the line-segment components and reduce the length of theline-segment components (NO of ST15, ST14). The controller 150re-executes the processing of extracting the line-segment components,based on the changed parameters (ST13).

On the other hand, when the extraction result is good, the controller150 executes approximation with a polygon forming a closed loop, andgenerates candidates for the windshield region, using some of theline-segment components extracted for each of the left and right images,as described above. Then, the controller 150 executes processing ofaligning at least one line-segment component between the left and rightimages, in the same manner as the above (ST16). Thereafter, thecontroller 150 measures and estimates the position of the vehicle 40 asdescribed above (ST17).

As described above, according to the present embodiment, the controller150 of the vehicle detection apparatus 100 includes a re-executingfunction (f2-1) of changing parameters used in extraction ofline-segment components based on the number and length of line-segmentcomponents for each extracted direction, and re-executing the processingof extracting line-segment components. This structure enablesimprovement in the extraction rate in the case of extracting theline-segment components of the windshield region, in addition to theeffect of the first embodiment.

The present embodiment can also achieve assured extraction of linesegments of the windshield region as described above, even with theconfiguration in which parameters used in extraction of line-segmentcomponents are changed based on the size of the windshield region andprocessing of extracting line-segment components is re-executed. Thepresent embodiment may be configured to use rectangle detectioninformation of the number plate detected upon vehicle entry, andperforming processing of correlation for information of four sides ofthe rectangle, when main line-segment components cannot be extractedeven when detection of the windshield region has been attempted aplurality of times with changed processing parameters.

Specifically, the re-executing function (f2-1) of the controller 150 maybe configured to include a first line-segment component extractingfunction (f2-1-1) of extracting a plurality of line-segment componentsforming the windshield region of the vehicle, and a second line-segmentcomponent extracting function (f2-1-2) of extracting a plurality ofline-segment components forming a rectangular region of the number plateframe of the vehicle. Together with this, the polygon approximationfunction (f3) of the controller 150 can execute polygon approximationprocessing using line-segment components extracted by the secondline-segment component extracting function (f2-1-2), when no closed loopcan be formed based on line-segment components extracted by the firstline-segment component extracting function (f-2-1-1).

The second line-segment component extracting function (f2-1-2) mayinclude extracting a headlight region of the vehicle by labeling from animage of the image data to be processed that has been subjected topreprocessing, and extracting a rectangular shape similar to the numberplate from a range estimated from the position thereof. Generally, anumber plate exists under the center between the left and rightheadlights and under a line connecting the headlights. According to sucha structure, the controller 150 executes polygon approximationprocessing using line-segment components extracted by the secondline-segment component extracting function (f2-1-2), when no closed loopcan be formed based on line-segment components extracted by the firstline-segment component extracting function (f-2-1-1). Thus, even whenline-segment components of the windshield region cannot be securelyextracted, the position of the vehicle can be measured by extractingline-segment components of the number plate frame.

Third Embodiment

FIG. 8 is a diagram for explaining a third embodiment. A vehicledetection apparatus 100 and a vehicle detection system according to thepresent embodiment are the same as those of the first embodimentexplained with reference to FIG. 1 and FIG. 2, and explanation thereofwill be omitted.

As illustrated in FIG. 8, the vehicle detection apparatus 100 of thepresent embodiment is configured to calculate a vehicle width 410 and avehicle height 420 of a vehicle 40. In the present embodiment, acontroller 150 is configured to include a measuring function (f4) ofmeasuring the vehicle position, which includes at least one of a vehicleheight calculating function (f4-1) of calculating the vehicle height 420and a vehicle width calculating function (f4-2) of calculating thevehicle width 410.

As illustrated in FIG. 8, the vehicle height calculating function (f4-1)calculates a height of and upper side of a windshield region 400 of thevehicle 40 as the vehicle height 420, based on coordinate informationbetween the line-segment components used for the measuring function(f4). The vehicle width calculating function (f4-2) calculates a lengthof a bottom side of the windshield region 400 of the vehicle 40 as thevehicle width 410, based on the coordinate information.

Specifically, the vehicle height calculating function (f4-1) usesinformation (coordinate information between line-segment components) ofline-segment components associated with each other between left andright cameras for line-segment components forming the windshield region400. For example, as illustrated in FIG. 3 (B), when the controller 150performs matching for vertical lines on the left and right side surfaceparts of the vehicle between the left and right cameras by the vehicleheight calculating function (f4-1), the controller 150 calculatescoordinate information of end points e1 and e2 of the line-segmentcomponent corresponding to the upper side of the windshield. Thus, thecontroller 150 performs matching for the line-segment componentcorresponding to the upper side of the windshield based on correlationbetween the coordinates, and thereby calculates the height 420 from theupper side of the windshield 400. The vehicle height calculatingfunction (f4-1) may include calculating the height of the upper side ofthe windshield, by changing the configuration of the camera parametersand directly determining the correlation of a difference (e1−e2) betweencoordinate information items of the end points e1 and e2 of the upperside of the windshield.

On the other hand, as illustrated in FIG. 3 (B), when the controller 150performs matching for vertical lines on the left and right side surfaceparts of the vehicle between the left and right cameras by the vehiclewidth calculating function (f4-2), the controller 150 calculatescoordinate information of end points e6 and e3 of the line-segmentcomponent corresponding to the bottom side of the windshield. Thus, thecontroller 150 performs matching for the line-segment componentcorresponding to the bottom side of the windshield based on correlationbetween the coordinates, and thereby calculates a distance |e6−e3|between the end points of the bottom side of the windshield 400. Thevehicle width calculating function (f4-2) may include calculating thedistance “(e6−e5)+(e5−e4)+(e4−e3)” between the end points of the bottomside of the windshield, by changing the configuration of the cameraparameters and directly determining differences (e6−e5), (e5−e4), and(e4−e3) between coordinate information items of the lower side of thewindshield.

Next, operation of the vehicle detection system including the vehicledetection apparatus 100 according to the present embodiment will beexplained hereinafter. The present embodiment is configured to addprocessing of calculating at least one of the vehicle height 420 and thevehicle width 410, in the processing of Step ST5 illustrated in FIG. 4.

Specifically, the controller 150 executes the steps of Steps ST1 to ST5illustrated in FIG. 4. In such successive processing, the controller 150measures the position of a closed loop as the vehicle position, based oncoordinate information between the line-segment components aligned inthe processing of Step ST4 and photographing position information(camera placement information) indicating the photographing positionwhere the left and right images have been photographed (ST5).

In this case, the controller 150 calculates the height of the upper sideof the windshield region 400 of the vehicle 40 as the vehicle height 420by the vehicle height calculating function (f4-1), as illustrated inFIG. 8, based on coordinate information of the line-segment componentsaligned in the processing of Step ST4. In addition, the controller 150calculates the length of the bottom side of the windshield region 400 ofthe vehicle 40 as the vehicle width 410 by the vehicle width calculatingfunction (f4-2), as illustrated in FIG. 8, based on coordinateinformation of the line-segment components aligned in the processing ofStep ST4. In the present embodiment, the controller 150 may execute oneof the vehicle height calculating function (f4-1) and the vehicle widthcalculating function (f4-2).

The controller 150 notifies the ETC system 30 of the vehicle position(coordinates) measured by the processing of Step ST5, and the vehicleheight 420 and/or vehicle width 410, through the network interface 140.The ETC system 30 transmits and receives wireless signals to and from anantenna of an ETC in-vehicle unit mounted onto the windshield, based onthe position (coordinates) of the vehicle 40 and the assumed passingspeed of the vehicle 40. In addition, the ETC system 30 can obtainadditional information of the vehicle height 420 and/or the vehiclewidth 410, and calculate, for example, statistics of the size ofvehicles passing through the ETC lane.

As described above, according to the present embodiment, the height ofthe upper side of the windshield region 400 of the vehicle 40 can becalculated as the vehicle height 420, based on the coordinateinformation. In addition, the length of the bottom side of thewindshield region 400 of the vehicle 40 can be calculated as the vehiclewidth 410, based on the coordinate information. Thus, it is possible toestimate additional information of the vehicle height 420 and/or thevehicle width 410, in addition to the effect of the first or secondembodiment.

According to at least one of the embodiments explained above, at leastone line-segment component is aligned among a plurality of line-segmentcomponents indicating the boundary between the windshield region and thevehicle body of the vehicle between the left and right images. Thisstructure can determine feature points which are correlated with eachother between a plurality of images, clarify the alignment standard, andimprove the vehicle detection accuracy.

Each of the methods described in the above embodiments can be stored anddistributed as a computer-executable program in storage media such asmagnetic disks (such as floppy (registered trademark) disks and harddisks), optical disks (such as CD-ROMs and DVDs), magneto-optical disks(MO), and semiconductor memories.

The storage media may adopt any storage form, as long as it is a storagemedium that is capable of storing a program and readable by a computer.

An OS (operating system) operating on a computer, and MW (middleware),such as database management software and network software, may executepart of the above processing to achieve the above embodiments, based oninstructions of the program installed in the computer from the storagemedium.

In addition, the storage medium in each embodiment is not limited to amedium independent of the computer, but includes a storage medium whichstores or temporarily stores a downloaded program transmitted through aLAN or the Internet.

The storage medium is not limited to one, and the storage medium in thepresent invention also includes the case where the processing in each ofthe above embodiments is performed from a plurality of media. The mediumstructure may be any of the above structures.

The computer in each embodiment executes the processing in each of theabove embodiments based on a program stored in the storage medium, andmay be any of a device such as a personal computer or the like, and asystem formed by connecting a plurality of devices through a network.

The computer in each embodiment is not limited to a personal computer,but also includes a processing unit and a microcomputer included in aninformation processing apparatus, and is a general term for apparatusesand devices that are capable of achieving the functions of the presentinvention by a program.

Although some embodiments of the present invention have been explainedabove, these embodiments are presented as examples, and are not aimed atlimiting the scope of the invention. These new embodiments can becarried out in other various forms, and various omissions, replacements,and changes are possible within the range not departing from the gist ofthe invention. These embodiments and modifications thereof are includedin the scope and gist of the invention, and included in the inventionsrecited in the claims and the scope equal to them.

While certain embodiments have been described, these embodiments havebeen presented by way of example only, and are not intended to limit thescope of the inventions. Indeed, the novel embodiments described hereinmay be embodied in a variety of other forms; furthermore, variousomissions, substitutions and changes in the form of the embodimentsdescribed herein may be made without departing from the spirit of theinventions. The accompanying claims and their equivalents are intendedto cover such forms or modifications as would fall within the scope andspirit of the inventions.

What is claimed is:
 1. A vehicle detection apparatus comprising: aprocessor, the processor being configured to: detect a vehicle from animage obtained by photographing the vehicle, the image including a leftimage and a right image that are photographed from left and rightdirections with respect to the vehicle; extract a plurality ofline-segment components indicating a boundary between a specific regionof the vehicle and a vehicle body and included in the image for each ofthe left and right images; and align at least one line-segment componentbetween the left and right images among the line-segment componentsforming a closed loop for each of the left and right images, and measurea position of the closed loop as the position of the vehicle based oncoordinate information between the aligned line-segment components andphotographing position information where the left and right images havebeen photographed.
 2. The vehicle detection apparatus according to claim1, wherein the processor is further configured to: execute a polygonapproximation by executing approximation with a polygon forming theclosed loop using some of the line-segment components extracted from theimage, for each of the left and right images.
 3. The vehicle detectionapparatus according to claim 1, wherein the processor is configured toalign all of the line-segment components forming the closed loop.
 4. Thevehicle detection apparatus according to claim 1, wherein left and rightphotographing positions of the photographing module are placed toinclude a windshield of the vehicle and side surfaces of the vehicle,and are arranged along a direction substantially parallel with alongitudinal direction of an upper side and a lower side of thewindshield.
 5. The vehicle detection apparatus according to claim 1,wherein the processor is further configured to: classify the extractedline-segment components according to directions, change parameters usedin extraction of the line-segment components based on a number and alength of the line-segment components for each of the directions toincrease the number of the line-segment components and reduce the lengthof the line-segment components, and re-execute processing of extractingthe line-segment components based on the changed parameters.
 6. Thevehicle detection apparatus according to claim 2, wherein the processoris further configured to: extract a plurality of first line-segmentcomponents forming the windshield region of the vehicle; and extract aplurality of second line-segment components forming a rectangular regionof a number plate frame of the vehicle, and wherein the processorexecutes the polygon approximation using the extracted secondline-segment components, when the closed loop cannot be formed based onthe extracted first line-segment components.
 7. The vehicle detectionapparatus according to claim 5, wherein the processor changes theparameters used in extraction of the line-segment components to reducethe number of the line-segment components and increase the length of theline-segment components when the windshield region has a size greaterthan a reference size, and to increase the number of the line-segmentcomponents and reduce the length of the line-segment components when thewindshield region has a size equal to or less than the reference size.8. The vehicle detection apparatus according to claim 1, wherein theprocessor is further configured to calculate a length of a bottom sideof the windshield region of the vehicle as a width of the vehicle, basedon the coordinate information.
 9. The vehicle detection apparatusaccording to claim 1, wherein the processor is further configured tocalculate a height of an upper side of the windshield region of thevehicle as a height of the vehicle, based on the coordinate information.10. A vehicle detection method applied to a vehicle detection apparatusdetecting a vehicle from an image obtained by photographing the vehicle,the image including a left image and a right image that are photographedfrom left and right directions with respect to the vehicle, and themethod comprising: extracting a plurality of line-segment componentsindicating a boundary between a specific region of the vehicle and avehicle body and included in the image for each of the left and rightimages; and aligning at least one line-segment component between theleft and right images among the line-segment components forming a closedloop for each of the left and right images, and measuring a position ofthe closed loop as the position of the vehicle based on coordinateinformation between the aligned line-segment components andphotographing position information where the left and right images havebeen photographed.